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A«?TBACT 

Inforsiation useful £or the initiation and operation 
of an individualized learning center CILC> is provided in this manaal 
which also includes an evaluation of the ILC for the Aaerican Indian 
students of the Cook School in Arizona. The reconaendations in the 
nanual are based on the experience gained in operating the ILC at the 
Cook School. The manual details in great depth aspects of an ILC; the 
processes of planning, staffing, and organizing an ILC are described 
along vith the important areas of institutional support, finance, and 
selecting objectives. The administration of an ILC and the processes 
of monitoring and grading stsdent progress are also described. The 
evaluation concluded that students using the ILC at the Cook School 
made statistically significant gains in reading vocabulary and 
comprehension, mathematics computation and problem solving, and 
language arts, but not in spelling. An attitude inventory indicated a 
positive feeling toward the ILC. The extensive appendixes contain 
many of the forms and descriptions of the material used in the ILC. 
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PREFACE 



When adult Indian students In a post-secondary education program are 
making three to five years acad«nic gains in such areas as reading, language 
arts, and wathematics in only three or four nwnths; when the average rate 
of attendance is over 98%; and when the grade point average for the institution 
is raised one whole grade point, in our opinion these results and procedures 
should be shared. 

The Individualized Learning Center at Cook Christian Training School f 
Tempe, Arizona has now completed its second year of operation. Thanks to a 
grant from Lilly Endowment, Inc., a lot of hard work by staff, and a consid> 

erable amount of student enthusiasm. 

This Manual has been developed for persons working in the areas of Indian 

adult basic education, college preparatory and/or remedial education, and church 
career development. Other persons may profit by using this Manual because an 
Individualized Learning Center is more than, and not necessarily dependent upon, 
a "place." Persons working in the area of secondary education among Indian people 
may find individualized instruction a useful alternative or supplementary in- 
structional system. 

As these two years come to an end, special appreciations are rightfully due 
to Lilly Endowment, Inc., for their cooperation and support; to Jeanne Smith, 
John Hogue, Roberta Yazzie, and Bill DeBoer, who as full or part time teachers 
have helped not only in the development but also in the operation of the ILC; 
to Carolyn Ray, who for a year and a half as a secretary, handled much of the 
paper work; to Or. Cecil Corbett, Executive Director of Cook School; and to the 
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other staff members who were trusting enough to give us a chance and supportive 
enough to make the ILC possible; and to the many visitors who at the right 
time {when we were discouraged) told us that the IIC had promise. 



David Campbell, Director 
Developanental Learning laboratory 

Gary Rush 

Dean of Instruction and 
Project Supervisor 

July 1974 



INTRODUCTION 



What do you do as an educator when you are faced with any c** all of the 

following conditions: 

1 . WIDE DIFFERENCES IN STUDENT ACADEMIC BACKGROUNDS. What do you do when 
students educational backgrounds vary from grade school to college so 
that the differences among students are greater than the similarities 
which make it difficult If not Impossible to have traditional classes? 

2. SCHEDULING DIFFICULTIES. What do you do when, because of the many 
student differences, as well as factors such as academic programs, 
employment, illness, and family situations make It a nightmare If 
not an impossibility to schedule the right classes for the right 
students at the right times? 

3. NEGATIVE STUDENT ATTITUDES TOWARDS THEMSELVES AND EDUCATION. What do 
you do when, because of past negative experiences in school, students 
are hesitant to learn because of their fear of failure? 

4. LIMITED INSTITUTIONAL RESOURCES. What do you do when there are in- 
sufficient funds and personnel to solve the above mentioned problems 
with usual educational methods? 

Cook School was confronted with these conditions and others. For us, 

"individualized instruction" provided In an •'Individualized Learning Center" 

helped us find a partial solution. This Manual was developed in order to 

disseminate our procedures and results with others In similar circumstances 

who might learn from our experience and/or want to replicate our experiment. 

Background of Cook School 



Cook School is a private, church- related, post-secondary institution 
which has for over 65 years prepared Indian leadership for the Indian church 
and community. Students have come to Cook School in the past from over 65 
tribes and from 27 states and provinces of Canada. The school, located In 
Tempe, Arizona, (a suburb of Phoenix, Arizona) is governed by a predominately 
Indian Board of Trustees which widely represent the various denominations and 
Institutions working in education among Indian people. 
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On the average there are 100 Indian students, 18 years of age or older, 
who are In residence at Cook School and of these students, approximately 50 are 
taking courses full or part time. In addition to the resident program. Cook 
School offers education by extension In various forms to an additional 1,500 
persons. 

The programs of Cook School are In three general areas: adult basic 
education, college preparatory, and ^-hurch career devf /lent. These three 
areas of programs, which are distinct for many Indian educational Institutions, 
are combined at Cook School. Cook School has, therefore, served as a useful 
laboratory for the development of educational programs which could be appli- 
cable to persons working in Indian adult basic education, college remedial 
education programs, and church leadership development projects. 

The Nature of the Problem 

In 1971 Cook School conducted an educational needs assessment using the 
Iowa Test of Educational Development to determine present student status in the 
various basic study areas. It was discovered that on the average, student 
abilities were at the grade school level despite the number of students who had 
completed high school and that the differences in backgrounds were greater than 
the similarities, thus making the traditional approach to education an inpossl- 
bility. While these measures were being taken, faculty and administrators were 
considering the educational levels which students should have for entering 
programs and at the time of completion of programs. It was decided by the school 
that students should have an eighth grade level of reading and language arts and 
a sixth grade level in mathematics before entering the school resident programs 
or beginning college studies. 

Once we had determined present student status and desired student status, 
we had to consider how, with limited resources, we could bridge the gap. We 



considered three basic options. 

The first option we examined was to "change the student body." It is at 
least tempting, if not possible, to consider changing the nature of the student 
body when their entry level Is far below the desired study academic level, but 
this option proved to be illogical. Studies have shown that on the average 
Indian students reach but the eighth grade and that their educational background 
as measured by standard tests is often bel<»# the eighth grade level. 

The second option we considered was to add more staff and classes, but to do 
this could have meant a student-faculty ratio of two to one (two faculty for each 
student) in order to have the number of specialists needed for the student body. 
By adding more staff and classes we would have proliferated the number of courses 
making it impossible to provide a logical schedule of classes. It was clear to 
us that the traditional approach to learning was £n impossibility as long as we 
maintained our primary concern for student learning. 

After considering the third option (to find other employment), we gave care- 
ful consideration to the feasibility of Individualized instruction and an Indi- 
vidualized Learning Center. After a careful review; of literature was done and 
many discussions were held, a proposal was conceptualized and submitted to Lilly 
Endowment, Inc., in the spring of 1972 and was funded in August 1972. 

The remainder of this manual explains and illustrates the questions which 
we considered, the answers to our questions, the steps which we went through, 
the resources which we used, and the results which we experienced. 

The manual is in two parts. Section I is, designed as a "How to do it" 
booklet for those who would like some answers to the problems indesigning, 
equippinq, and operating an Individualized Learning Center. 
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Section II contains a description of the evaluation procedures used to 
answer certain questions we had about our particular learning center and the 
student population we served. 

Since the sections may be considered or used independently, we have 
provided a separate appendix for each. 



SECTION I 



PLANNING* STAFFING AND OPERATING 
AN INOIVIDUAtlZED LEARNING CENTER 



PLA?|f'IflG. STAFFING, AND OPERATING A.i IflDIVIOUALIZED LEARNING CENTER 



OBJECTIVES 

So««e of us who had been singing the praises of individualized instruction 
for years suddenly found ourselves in a position to start practicing what we 

had been preaching. 

One happy day in August 1972, we learned that Lilly Foundation was funf*ing 
our plans for an Individualized Learning Center. 

Ue discovered ouite early that novino from the concepts and theories of 
Individualized instruction to the nuts and bolts of actually putting a center 
into operation is a sizable step. We longed for a comprehensive "how to do it" 
nanual. If there was such a publication around, we never found it; so we have 
vwitten one of our own. 

It occurred to us that there may be others who want to put together a 
learning center and who would appreciate sone suggestions fron those whc h^' 
been there. 

Our exoerience in designing, staffing, equipping, and operating an 
Individualized Learning Center (I.L.C.) is linited, and therefore so is our 
expertise. Moivever, if we had known two years ago what we know now, our trip 
would have been a lot easier. He don't have all the answers; but we do have 
sone, and we would like to share them with you. 

The objective for our nianual is that those who use the manual will: 

1. Know the major steps involved in designing, staffing, equipping, and 
operating an I.L.C. 

2. Be able to accomplish the necessary first steos of establishing 
institutional and faculty support for this approach to learning. 
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3, Be able to produce a tiow chart or oroqran calendar as a guide for 

rakina decisions anc neetinq deadlines. 
^. Be nore knowledqeable about designing, building, and/or purchasing 

furniture and equipnent. 

5. Have a list of learninc packages which we have found to be successful 
(and sone which were not). 

6. Have exanples of sone tested procedures for: 

schedulina students 
writing contracts 
ronltoring student progress 
oradinn 

reporting to students 

7. Have a description of sone evaluation rrocedures, including the 
Instruments which were used. 

8. Have sone sample attitude surveys for students and staff. 

9. Have sanples of some of the many foms (instructions, recording data, 
student progress, etc.) we found helpful. 

10. Have an estln^ate of the time and costs involved in accomplishing 
these things. 

INSTITUTIOriAL SUPPORT 

Putting the nost rodest I.L.C. Into operation is impossible without the 
support of those who n-ake the decisions about capital outlay, budget, and 
personnel. This Institutional support nust necessarily be more than a nodding 
approval of an Idea. It will usually involve a finr. conwitment on money, 
staff for planning, space, and time. To the extent that other teaching staff 
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will be involved or affected, their approval is certainly Important. 

Whether you are a faculty member going through the "proper channels" to 
the board or a spokeswan for a group of interested parents, you will need to 
do your homework. This homework, or planning, will be much the same in 
either case. 

After offering some convincing (we hope) arguments for this approach, 
you should be ready to offer some practical suggestions about how to do it. 

Why an I.l.C? 

Mere are some benefits from this approach to learning which might be 
used as arguments, if needed. 

1. We all pay lip service to "individual differences" In students. 
Our traditional "cells and bells" approach to schooling, however, 
makes it nearly impossible to do anything about it. 

2. An Individualized Learning Center allows for individual differences 
because: 

• students enter a learning sequence fft their level of ability 
and understanding, 

• students move through instruction as fast as their Interests 
and abilities permit, 

• students exit when and if they have demonstrated mastery, not 
because the period Is over or the teacher's lesson plan dictates 
that the next unit must be started, 

• an instructor and/or teaching aide in an I.L.C. can provide 
personal help to a student when he or she needs It. This Is not 
always possible in the traditional classroom. 

o 11 
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• the tensions of con^etitlon (and the inevitable falling grades for 
many) are reduced. It is now possible for almost all students to 
succeed by making time the variable. This simply takes advantage 
of the fact that any reasonably normal youngster can master the 
tasks required In a typical public school. It just takes some 
longer than others. 
Some additional benefits - 

3. Students will have more opportunity to practice those behaviors we 
label "self-motivating," self-propelling," and "self-directing." We 
do want this, don't we? 

4. An I.L.C. allows for great flexibility In scheduling not possible In 
the "cells and bells" routine. 

5. The number and variety of courses or subjects offered Is limited only 
by the availability of well -designed Instructional systems - not by 
the availability of additional rooms and teaching staff. 

6. Because the chances of failing are reduced, the needed to cheat to 
survive or succeed is also reduced. The student's grade Is determined 
by what he or she accomplishes as he or she works through the material 
not by how he or she compares to others. 

7. The number and quality of programmed and other kinds of self-instruc- 
tional products is increasing. 

8. And last, but certainly not least, this approach to learning seems to 
make a lot of sense to students. This is the case with the older 
students we have surveyed. 

Arguments such as these should make sense to board members, parents, teach 
ing staff, students, or anyone else who believes that schools should be in the 
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business of learning. 

An I. I.e. cannot do everything. One important kind of learning is that 
which occurs during group interaction. This is not easily acconuiodated in an 
I. I.e. and must be provided for elsewhere. 

Personnel 

A commitment to start planning an I.t.C. should include at least one 
staff person full time for this effort. Others will be involved part time as 
particular talents are needed (such as someone to do the working drawings for 
remodeling an existing space or designing a new facility). 

Tactics 

To gain staff support and understanding* a visit to an I.L.C. may be 
helpful. There are those who seem to have trouble dealing with concepts and 
need to see an actual center "In action." If this first-hand experience is 
necessary, it may be accomplished in two ways. You might make arrangements 
with the director for a "tour." Or, you might take a portable TV camera or 
still camera and a cassette recorder and bring back an audio (Interview) visual 
report to show to staff members and parent groups. We used this latter approach 
and found it to be most effective In building staff understanding. Be sure to 
let the director of the center you are to visit know what you are planning. If 
the director and his or her staff are expecting you, you will probably find 
that they are more than anxious to "show and tell." Remember to go with a list 
of carefully prepared questions for your Interview. 
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Cost 

How much does an I.L.C. cost? Obviously, there Is no simple answer to 
this. There are some estimates which can be used for planning, however. 

For an I.L.C. of from 20 to 25 student stations, a figure of $400 per 
station win work pretty well. This figure will Include costs of furniture, 
machines, software, and office equipment. This does not Include costs of 
remodeling existing space or building a new facility. Remodeling an existing 
classroom might be done for $2500 and would Include carpeting, adding electrical 
outlets, and partitioning for office and work space. 

We based our estimates on the following costs: 



3 tables, 3'x6' (12 stations) ?$150 450 
13 carrels, "wet" @$125 1600 
25 chairs @$12 300 
8 deck-mounted cassette players (?$120 960 

4 machines 0$2OO 800 
1 video tape player 1200 
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\ TV monitor 



150 



Office furniture and equipment 



1500 



Shelving (purchased or built in) 
Software (learning packages) 



1500 



400 



$8860 



See appendix for drawings of learning centers. 

The two figures which would vary the most are software and machines. What 
you spend in this area depends, of course, on how many skill areas you plan for, 
the range of materials (grade levels), and the hardware involved. 

These various elements will be discussed in more detail later. 

How much planning time? 

An I.L.C. may be as simple or as ccmiprehensive as your need and resources 
dictate. It may be set up to deal only with improving reading skills, or it 
may be designed to improve skills in many subject areas. 

If a number of decisions have already been made (number and kind of students 
and learning objectives identified), one semester should be enough for planning. 
The physical space for the center can be readied while the staff planner selects 
and orders the necessary instructional materials and equipment. 

If, however, you are starting with only the idea that some kind of I.L.C. 
might meet some of your needs, you have much to do. In this case, we would 
recommend a full time staff member for the full school year. This person 
would gather data about I.L.C. 's and meet periodically with other staff to 
follow through the steps outlined in this manual. 
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Planning Tools 

Planning calendar (see Appendix G) 

Using a planning calendar can be very helpful In keeping you on the 
track and meeting deadlines. 

A planning calendar, and the charts used to display a planning calendar, 
spell out - 

what tasks are to be accomplished 
the order In which they occur 
what persons are to perform these tasks 
when each task Is to start 
when each task Is to be completed 
Putting this down on paper and making copies available to all who are 
Involved In the planning has many advantages which should be obvious. 
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We have Included a sample planning calendar and chart used to display the 
various tasks. You will note that some activities must follow others but many 
can overlap or be occurring at the same time. 

Flow Chart (See appendix H) 

The flow chart which we used in planning our I.L.C. Is Included to give 
you another example of a planning tool which might be useful. 

The one element which makes a flow chart different from a calendar Is the 
"decision block." These are a series of critical conditions which must be met 
before other activities can proceed. These conditions are In the fonn of 
questions which can be answered "yes" or "no." If "yes," you proceed to the 
next activity; if "no," what then? The flow chart should show what happens 
then. 
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Anticipating possible problems and planning alternatives will reduce the 
number of "surprises." Surprises are fun on your birthday but not while you 
are thrashing your way toward a deadline. 

Instructional objectives 

This discussion will reflect our peculiar bias about instruction. We 
assume that our obligation as educators is causing and evaluating learning , 
not merely providing instruction; there is a difference. 

If you accept this discipline, you accept the obvious implication that 
you cannot make intelligent decisions about kinds of Instruction and materials 
until you have determined what specific skills are to be learned by what kinds 
of students. 

We do not see an I.L.C. as a study hall, as "supplemental" or for 
"enrichment" or as a novelty to demonstrate that we are into the latest 
educational fad. We do see the I.l.C. as a very effective vehicle for causing 
certain kinds of learning. 

What has all this to do with planning? Simply this: you cannot design a 
facility to accomplish "X" until you have defined "X." 
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learner performance objectives 

Let's start by Identifying the kinds of learners who will be using the 
center. Identify how? By age? By grade level? By demonstrated needs In 
particular skill areas? The last would seem to make sense. 

Nextt determine those particular skills to be learned or Improved which 
your I.L.C. will deal with and the range of abilities to be accommodated by 
the instructional materials. 

"Range" refers to entry and exit levels in any kind of Instruction which 
is sequenced, such as instruction in reading. For examplet will your self- 
instructional materials in reading range from 4th through 8th grade* from 7th 
through 12th, college level? 

By now, most of us are familiar with the form of "behavioral" or "learner" 
objectives. These "performance" objectives described what the learner Is able 
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to do ds a consequence of Instruction. In order to define entry and exit 
levels for the various areas of instruction, it is necessary to write 
instructional objectives in this fomi. 

For example, "The student will demonstrate a reading level of grade 
or better as measured by (test) 

This criterion measure may be used as a prerequisite behavior for entering 
or exiting an instructional unit or program. 

One of the most commonly used methods for describing levels of academic 
achievement is "grade level." Although not precise, using grade level has some 
practical advantages, ftany publishers of self -instructional materials describe 
the range of their products this way. Many standardized achievement tests 
yield scores which translate into "grade level." This is also a term which 
seems to be generally understood by the layperson. 

What constitutes "Mastery?" 

For most cognitive learning, a quantitative definition of mastery Is the 
performance level (score) on some kind of test. Whether "mastery" performance 
shall be lOOfo 95%, 90%, or something less is an arbitrary decision based on 
the particular importance attached to the learning Involved. When a particular 
skill is an essential prerequisite for subsequent learning, complete mastery 
is required. Stated another way, everyone must get an "A" in the unit or 
program before he or she proceeds to subsequent units. 

Remember, we are not interested in developing bell-shaped curves; we are 
interested in causing learning. There is no need to continue to demonstrate 
that there are individual differences In aptitude for learning: this has been 
pretty well established. 
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We are aware of the many criticisms of standardized achievement tests 
(norm- referenced tests). However, until such time as there is a consensus 
about what, precisely, constitute the essential academic skills, how they are 
best taught/ learned, and how they are measured, we will suggest that these 
tests provide a useful estimate of student learning. 

An alternative, of course, is to develop your own specific criterion 
neasures and try to natch instruction to these. This Is a formidable task. 
For example, the Instructional Objectives Exchange book lists 313 objectives 
in the area of reading for kindergarten through third grade. 

Matching specific instruction to each of these would require a great deal 
of time. We doubt that the typical school system Is prepared to commit 
resources to this kind of effort. Therefore, our model for a learning center 
is more modest. It does not answer all of the criticisms of our traditional 
approach to schooling, but it does satisfy some of the more obvious ones. 

SELECTING INSTRUCTIONAL mTERIALS 

The materials used in an I.L.C. are self -instructional (auto-tutorial ). 
Any book we might read and learn from could be considered "self-Instructional" 
In that sense. However, we are talking about something more than this. 

Criteria 

To meet our definition of "self-Instruction," a learning package or 
system must meet a number of criteria: 

• it should have pre- and post- tests to monitor and measure student gain 

• it should have diagnostic or prescriptive tests to determine student's 
present status and entry level into the material 

• it should have learning objectives stated in performance terms 
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• It should have criterion tests to measure the achievement of these 
objectives. 

Most commercial materials which are offered fail to meet these criteria. 

After you have made some decisions about skills to be learned and have set 
some limits as to range, you come to the fun part. "Which one shall we buy?" 

Selecting particular kinds of self- 
instructional materials from the myriad 
of stuff offered for sale was the most 
challenging and time-consuming part of 
putting together a learning center. 
Apparently most manufacturers are able 
to sell their wares with a glib sales 
pitch and attractive packaging. It is a 
rare company and/or salesperson who can 
(or will) furnish validation information 
containing hard data on the effective- 
ness of their materials. Happily, the pressure is on the producers of instruc- 
tional materials to become more accountable. (Note the new California and 
Florida laws requiring publishers to shw some evidence that their educational 
materials will do what they claim.) 

See appendix 0 for a list of some materials we have used and would 
recommend. 

Individualized vs personalized 

The materials we use are individualized only to the extent that a student 
works through them at his or her own level and at his or her own rate. Secause 
of his or her performance on a diagnostic pre- test, the student may be working 
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In only certain oarts of the material. In the more precise sense, the 
materials are not really individualized; that is, they are not personalized 
or designed for that student alone. See instructional model next page. 

What instructional mode is most effective? 

This is like askina which holds more, a bag or a box? So far as we have 
been able to determine, there is no one "best way" to present Instruction. A 
well -written programmed book in math can be more effective than a poorly- 
dc'>igned program using sound/ films trips, color slides, or films; and it is a 

lot less expensive. 

Our learning center, which emphasizes training in basic academic skills, 

uses the following modes: 

printed programmed materials 

sound/filmstrips 

audio cassette tapes 

video tapes 

sound/slide 

Most of our materials are in a printed, progransned format. 

Many of our learning packages use more than one mode. For example, in our 
study skills course (six units), we use progi^afflmed text, video tape (a tour of 
the library), and audio cassettes (for listening-notetaking). Some particular 
skills obviously dictate the mode to be used. Improving skills In listening 
and notetaking require that the student listen to lectures or discussions 
while he or she practices organizing and writing down in appropriate form what 
he or she is hearing. You don't develop this skill by reading about it; you 
learn to listen and take notes by listening and taking notes. 
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BASIC INSTRUCTIONAL MODEL IN AN 
INDIVIDUALIZED LEARNING CENTER 



SELF- INSTRUCTIONAL COURSE 
IN LEVELS OF DIFFICULTY 



STUDENT ENTERS COURSE 
AT HIS OR HER LEVEL 



S1UDENT D 



r 



TEST T3 

OETERf'.INE ^ 
PRESENT LEVEL ^ B 
OF STUDENT 



r 



/ t 




i 



pre7Uj! |,J,^ I 



TEST 



ZIT 



j 



RANGE OF MATERIALS TO 
ACCOMMODATE EXPECTED 
ENTRY LEVELS 



TIME VARIES 



Time to reach mastery level will vary depending on: 

1. where student enters the learning unit 

2. how many hours per week the student spends 
working in the unit 

3. student's aptitude for learning 

4. student's motivation 
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Range and levels of naterials 

You may be planning your I.t.C. for elementary, high school, or college 
level students or for adults. Reading materials designed for elementary 
students in the fifth grade will probably not be appropriate for use by adults 
reading at the fifth grade level. There are self-instructional reading 
materials inade for both groups. 

PHYSICAL FACIIITIES 

Just as there is no one best Instructional mode, there Is no "best way" to 
layout a learning center. We can discuss some options and make some suggestions, 
however. 

A typical learning center will require these elements: 

1. Student stations 

a. tables (which seat four) 

b. carrels purchased or built In (some with cassette players) 

2. Open shelving, adjustable, for Instructional materials 

3. File cabinets (2-drawer) 

a. one for student folders 

b. one for Instructors (tests, answer keys, manuals, forms, etc.) 

4. Instructors' station (desk, storage) 

5. Private area (office), sound proof (counseling, visitors, phone calls, 
etc. ) 

6. General storage, supplies 

7. Work area (optional) for Instructional product development. 

In planning space requirements, the ideal situation Is to start with 
instructional objectives. These determine the kinds of learning programs, 
packages, and systems to be purchased. These, in turn, dictate the kinds of 
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space needed to accoirriodate them. See appendix for suggested layouts for 
remodeling a typical classroon. 

General considerations 

If there is good general lighting throughout the roop;(s), it is not 
necessary to have lighting in individual carrels. Carpeting is a must for 
reducing noise level, for maintenance, and to make the center more attractive. 

How many student stations? 

We would suggest 25 as maximum fnr any one area or room. This is perhaps 
as many as an instructor or tutor can keep up with. Keep in mind that in an 
I. I.e., at any one time students will be working at different levels in many 
different programs. There is no way to predict who will need what kind of 
help or when. 

Carrels or tables? 

We use both. After observing our students for a year, we discovered that 
they are equally divided as to their preference for a place to work. If the 
learning package requires the use of special equipment, the student will go to 
the machine. If a prcgranmed text is being used, the student may elect to work 
in a ftudy carrel or sit with others at a table. Carrels provide more privacy, 
which is important to some students. 

Audio cassette players 

We recofwnend that such units be playback-only and that they be built into a 
carrel if possible. Playback-only eliminates the possibility of tapes being 
erased. Having units built in ("deck mounted") eliminates the problems of 
storing, checking in and out, damage from handling, and having them "walk off." 
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Deck-mounted units are available from Avid Corp. (model 505 LC) and from 
Wollensak (model 2505 AV). 

How many? 

This, of course, depends on how raany student stations you plan for your 
I.L.C. and how many of your programs will use audio tapes. As a rule of thumb, 
we would reconmiend that up to one third of the stations be "listening stations." 
Since the students will be using headsets, you will not need speakers with the 
cassette players. See sketches for layout suggestions. 

"yiewinq stations" 

We use a sound filmstrip projector, a sound/slide projector, and a video 
tape player and monitor. Vie felt that the problems involved in trying to 
operate a 16mm projector in our I.L.C. ruled out any benefits. If we need a 
film, we put it on video tape and use it with our video tape player, 

Vihat kinds of tnachines? 

Because of our limited experience In using projectors, we are really not 
qualified to recommend specific models. There Is an organization which does 
this, however. "E.P.I.E.," Educational Products Information Exchange, (see 
appendix) is a sort of Consumers' Union 
of the edication business. In a special 
E.P.I.E. report (No. 60, March 1974), 
the cassette sound filmstrip viewers 
are evaluated. The standard LP 10-2 
is recommended. Prices of the machines 
evaluated ranged from $129.50 (Singer) 
to $295.00 (Dukane). (see appendix J) 
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We are involved in developing some of our own Instructional products so 
we purchased a sound/slide orojector (3M model 625 Sound on slide projector 
recorder). This machine will allow you to put together a learning package 
with color slides and accompanying narration which can be updated or modified 
easily. This is an expensive machine, but could be justified if you have 
time to develop materials for it. 

Most of these machines can be used in a carrel set up for that purpose. 
It is easier to have the student go to the machine than to be lugging the 
machines around. 

See the accompanying sketches for layout suggestions, (See appendix N) 

Storage of instructional materials. 

The first consideration is to make the materials accessible to students. 
To allow for growth of your I.L.C., It is best to make provisions for lots of 
adjustable shelving. 

At this point, a picture is probably worth many words so we have made 
sketches of some ways to display materials that have worked well for us. 




SEE APPENDIX FOR DETAILED PLAN 
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Need for separate roow 

There are many times during the day when the director and/or instructorCs) 
of the I*L.C. are involved in conversatlor« other than tutoring. The phone 
rings» another staff member comes by, a private conference with a student is 
needed, a salesperson drops in to show you that new unit in math, "visiting 
firemen" want to see and talk to you about your I.L.C, etc. It has been our 
experience that students resent these interruptions, particularly if they 
take place in the study area. It is important, therefore, to provide a 
private area where these activities can take place. It is also important 
that the space be close by and somewhat sound proof but not isolated visually. 
A glass panel in a separating door will take care of this. 

Staff work area 

Whether this is necessary depends on the extent to which staff members 
(I.L.C. staff and others) will be developing their own self-instructional 
materials for use in the I.L.C. 

If the bug gets you, it on be fascinating work. It takes a lot of time 
to do properly, however. The discipline involved in developing self- 
instructional materials which include specific performance objectives and 
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pre- and post- tests which measure the achievement of these objectives may be 
foreign to many teachers. 



Equ1pn%nt for instructional product development 

Again, the need here will be determined by staff interests and talents, 

budget, student learning needs, and time (the most precious resource of all). 

We have experienced the frustration of having a list of learning packages to 

do and the space and equipment needed but not the time to work on them. 
Assiffiiing that you have the time and plan to get into this, you will find 

these items helpful: 

chalk board planning board 

drawing board desk, table, or counter 

cassette recorder typewriter(s) 
machine for copying machine for binding 

ADMINISTRATION OF THE I.L.C. 

Up to now we have not defined the kinds of students the I.L.C. might be 
serving. 

If your I.L.C. is for adults, certain procedures will be different than 
if you are part of an elementary or high school. Elementary or high school 
students may be assigned to work in the I.L.C. and may have very little choice 
as to subject areas and the hours they will be in attendance. For adults, the 
I.L.C. will almost certainly have to be open several evenings a week, and the 
scheduling will be more flexible. 

Scheduling 

Because of the limited number of stations, students will need to be 
scheduled in the I.L.C, 
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One of the great advantages of an I.L.C. is the flexibility it offers in, 
scheduling students. For any particular course, you can schedule a student 
for from one to ten hours per week (or more If really necessary). Three to 
four hours a dav In an I.L.C., however, should be maxinnjm for a student. The 
number of hours scheduled per week may be dictated by the amount of material 
to be covered in a six-week or semester period or by the number of other 
courses the student is taking. 

Although this approach does not require that all students be at a certain 
level or place at a certain date, there should be some minimum acceptable rates 
for moving along through a program. Not all students will move at the same 
rate, but all should be making some progress. 

We use a large sheet {30"x40") made out as a calendar week with a block 
for each period. The names of students are written into each block of the 
period for which they are scheduled. See sketch. 




Another advantage to an I.t.C. is the ease with which students can make 
up or put In extra time. Unless every station is filled every period, a 
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student should be able to conie Into the I.L.C. at his convenience to do extra 
work. 

Combined I.L.C. and group classes 

Ue have some courses which Involve both study In the I.L.C. and group or 
seminar classes. For example, a student in an English composition class might 
meet with a group two hours per week and be scheduled in the I.L.C. for three 
hours per week. This assunffis, of course, that the two activities would be 
planned and correlated. 

MONITORING STUDENT PROGRESS 

Ue have two ways of monitoring a student's progress. Each student has a 
file folder for each course or study unit which he or she is working In. 
Attached to each folder Is a form whicli notes the days and hours the student 
is assigned to work In that unit. There is also a place for an entry to be 
made by the student after each study session in that unit, noting time In, tine 
out, and what work was done. See a typical entry on the sample below. 



SCHEDULE CONTRACT 




INDIVIDUALIZED LEARNING CENTER 

sr r /CO sw0vsKiU.s 3 

Course # Course Title Credit Hours 



Student's Name 

Days (Circle) (^nda^ Tuesday ^^ednesday) Thursday Q ^iday^ 
Hours (Fill in) f/^to/^/^g> to to jOt 00 to ^tOOtoiOl^O 



student s Signature 



Date 



Date 


Time 


Work Done 


Time 
Out 


Sovr's 
Initials 


Coimients 
















^/ 


ViB 



























STUOENT-S FOLDERS ARE FILED IN I.L.C. ^^^^ 



SCHEDULE CONTRACT STAPLED 
INSIDE FOLDER 




Ater makino an entry on the schedule contract form, the student leaves 
the folder in an "in basket" on the instructor's deck. During or at the end 
of the day, the instructor checks through the folders and notes where the 
student Is in the unit and how much is being covered during a period in the 
I.L.C. If the unit includes a booklet in which the student records answers 
to questions and problems and scores of en-route tests, this booklet !s also 
kept in the folder. This provides a second record of progress. In most self- 
instructional learning packages, the student scores his or her own en-route 
tests over the sections in a unit. The instructor will administer and grade 
a pre- or post-test over the unit. The score on the unit post-test will 
determine whether the student goes on to the next unit or must review and 
retake the test. 

Why pre-tests? 

The answer to this should be obvious. If a student can demonstrate on a 
pre-test that he or she already knows the data, principles, concepts, operations, 
or whatever to be covered in a unit, what purpose is to be served by making him 
or her po through the unit? Unlike the mountain climber, we don't require the 
student to do the unit "because it's there." 

A student works in a learning package or unit because his or her perform- 
ance on a pre-test indicates that he or she lacks a particular skill that the 
unit is designed to help him or her acquire and one which we have previously 
determined is important. It may be important in and of itself or as a pre- 
requisite for subsequent learning. 

Being able to skip over units also has certain motivational value to 
students. This is one example of what is meant by the phrase "...moving 
through the materials at their own rate." 
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Some pre-tests are designed (or can be inodlfled) to be diagnostic and are 
used to prescribe which sections a student is to study In a unit and which 
can be omitted. See the explanation of the Heywood math program as an example 
of this. (See appendix Q) 

ConsumafaTe materials 

Many of the commercial programmed texts are designed for the student to 
write In and would therefore be consumable. Buying 20 or 30 new copies of 
these every year or semester can be expensive- A student can just as easily 
mark his or her responses on a cut, lined, yellow pad or on a printed response 
sheet which can be run off on a copying machine. S«ne materials, such as 
S.R.A.*s kits, use "Student Record Books." These provide places to mark 
answers to each lesson and to record scores of the tests taken. These are not 
expensive and are therefore not worth trying to replace with sme substitute 
of your own. 

GRADING 

Administratively, we have found this to be biggest headache. If your I.L.C. 
has to follow certain traditional procedures, such as grade reports every few 
weeks or at semesters, this can present problems. 

If we sort students on the basis of the amount of work covered during a 
certain time interval, we are back In the same old bag of comparing students, 
which contradicts what we are trying to do. On the other hand, If we "grade" 
only the quality of work and not the quantity, we could say that everyone gets 
an "A." There is nothing wrong with this except that It doesn't seem quite 
fair to record as an "A" a student who got through only two units during the 
time that another student successfully completed eight units. 

i» 
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The belief that a "report card" with A's, B's» C's, O's, and F's on it 
really tells us something about what a student has learned is a belief that is 
largely a matter of habit and tradition. A report card which notes a "B" in 
math really tells us nothing about which math skills a student has mastered, 
if any. Most attempts to change this» however, are stubbornly resisted. 

The best answer is simply a progress report describing where the student 
is. If comparisons must be made, the student's present status could be con- 
trasted with the desired or expected levels to be reached. That is, his or her 
performance is compared to a criterion performance level, not to the perform- 
ance of other students. 

TESTING AND EVALUATION 

The Achievement Battery 

We use the results of these tests in two ways. First, they serve to deter- 
mine the present status of the student. Deficiencies, if any, are noted and 
work in the I.L.C. is prescribed. We uso a test which yields grade equivalent 
scores which makes placement in the materials easier. Secondly, giving the 
alternate form of these tests at the end of the year serves as one measure of 
the effectiveness of the learning center. 

Most achievement batteries test in a number of sub-skill areas. This allows 
us to be more precise in scheduling. For example, a student is not enrolled 
in "math," he or she is assigned a particular kind of math at a particular 
level for a specified number of hours per week. Or, for language arts, the 
student is not enrolled in "English" but is scheduled a certain number of 
hours in punctuation or grammar or vocabulary or composition or whatever the 
test indicates his or her need to be. 
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This matching of instruction to student needs assumes, of course, that 
self-Instructional materials at the needed level are on hand. 

If your I. I.e. Is part of a school system, giving these achievement tests 
to I.L.C. and non-I.l.C. students might provide data for comparing the 
effectiveness of what you are doing. Don't be afraid of these ccai^risons. 
You will probably come out looking good; If you don't, you should be the first 
to know and the first to want to do something about It. 

We have noted gains of several grade levels In some skill areas after a 
student has spent only one semester In the I.L.C, 

Other tests 

Most of our self-Instructional materials have their own tests. Many have 
small en-route tests which the student takes and grades himself or herself. 
Almost all have tests over major units which are a<fci1n1stered and scored by an 
Instructor. 

AHITUOE SURVEYS 

Do your students like this approach to learning? Why not ask them? We 
have Included a copy of a survey questionnnaire we have used for the past 
couple of years. The reason for such data gathering is to Improve what you are 
doing. Keep the questionnaires anonymous; hopefully, the students will let 
you know what they really think. (See appendix E) 

EVALUATION BY STUDENTS 

The real test of the usefulness and effectiveness of any instructional 
material or procedure is whether students can and will use It and learn what it 
was designed to teach. 

For those materials we have developed, we continually solicit student 
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feedback. Our first usable draft of a learning package we call a "prototype.** 

It is offered with ti^e expectation that it can and will be revised and 

improved. We encourage students to give us their coninents about the material 
on forms designed for that purpose. See appendix F. 
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APPEND. A 

INDIVIDUALIZED LEARNING CENTER 



Student's Name 
Days (Circle) Monday 
Hours (Fill in) ta__ 
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Date 


Time 
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SELF-INSTRUCTIONAL MATERIALS 



This listfng Is in no way comprehensive. It includes only those pro- 
ducts we have used and recojinend. There are many others available in 
these and other areas. 

1. READING 

S.R.A. (Science Research Assoc.) makes "reading labs" whose levels 
range from elementary through high school. We have used these "labs:" 
33, Illb, IVa, and R.F.U. (Reading For Understanding). These are 
printed, programmed materials designed to improve reading rate and 
comprehension. 

Craig reading programs require the use of reading pacers. These 
machines run more than $250 each. We have used reading programs "A," 
"B," and "V.P.R. (Vocabulary, Preview, Read)." 

R*A..S. ( Reading Attaimnent Systems— 1 and 2) are sfmilar to the 
S.R.A. kits in fonrat and ube. The essential difference is that the 
content of the reading materials is at an adult interest level. 

Basic Skills System (McGraw-Hill) consists of "study-type reading 
kits" in natural and social sciences. These are on the tenth grade 
reading level and are designed for college or college-bound students. 
The content and questions require reading for information and special 
facts. The Basic Skills System also includes a unit on listening and 
note- taking, which uses audio tapes and response books. We have 
included this as one unit of our study skills course. 

2. LANGUAGE ARTS 

" Word Clues " by E.D.L. (McGraw-Hill) is a vocabulary building pro- 
gram. The series consists of seven books for reading levels seven 
through thirteen. Placement tests are available. Hand-held mechanical 
tachistoscopes for rapid word recognition and spelling are used with 
programmed work books. 

GRAMMAR AND USAGE 

English 2200, 2600. and 3200 are produced by Harcourt Brace and 
Jovanovich. These are three progranmed books on grammar and usage, 
capitalization, punctuation, and ways of improvino writing style for 
junior and senior fiigh school levels. 

Basic Skills in English by Individual Learning Systens is senior 
high or junior college level. The I.L.S. materials are a series of 
printed, programmed texts which can be used several ways. Because of 
diagnostic tests, each student may take dn individual path through the 
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materials. The course is designed to teach students to write a cmpo- 
sition which is well -organized and correct in grauraar, punctuation* 
and spelling. The course includes a supplementary unit on listening 
and note- taking. 

PUNCTUATION 

We have used Punctuation; A Prosranmed Approach by Southwestern 
Publishing Co. This covers 27 basic rules of punctuation, using a 
printed, programmed text. 

3. MATH 

A First Program in Mathematics by Arthur H, Heywood (Wadsworth 
Publishing Co.) is a programmed approach dealing with math skills in 
five areas: whole nun^ers^ common fractions, decimal fractions, 
percent, and practical algebra. It is difficult to attach grade 
levels to these skills. Perhaps it would be safe to say that these 
materials are appropriate for middle elementary students, starting 
with Unit 1. Whole Numbers. 

Each unit has a diagnostic pre- test which permits a student to 
plan an individual path through the unit, studying only those elements 
he or she needs. Thi.f. flexibility in prescribing instruction adds tr^ 
the bookkeeping but does individualize a student's course of study. 
Bee Appendix Q for the forms we have developed to allow for prescribing 
(programming) and monitoring a student's progress through the units j7 

There are other programmed approaches such as those published by 
Grolier and by Sullivan. These are perhaps simpler to use but are not 
diagnostic and prescriptive. 

Basic Mathematics books I, II, and III, published by Westinghouse 
Learning Corp. are designed to teach addition, subtraction, multi- 
plication, and division of fractions; the basic operations with decimals; 
and basic principles of geometry. This program would seem to be useful 
as review or remedial for high school age or adult students. Tests 
for each section are diagnostic and prescriptive. 

4. STUDY SKILLS 

As yet, we have found no commercial package which satisfies our 
needs. We developed our own six-unit course which includes two units 
which are commercially produced. These units are the S.R.A. kits on 
graph and picture study skills and map and globe skills. Also, we have 
used a college-level program. How to Survive in College , by Education 
Marketing Corp. The course is on eight cassette tapes, together with 
a work book. 

5. CONSUMER EDUCATION 

We have used Modem Constaner Education , produced by Grolier. The 
material includes cassette tapes, film strips, booklets, and student 
response books. Units include ways to shop and handle money; getting 



a lawyer; you and your landlord; and buying furniture, appliances* a 
car. drugs, medicines, health insurance, etc. A n&t supplement i^dds 
five areas. A films trip projector which advances the film on an 
Inaudible cue is reconmended. 

In this area, a new booklet. Forms in Your Future , gives the 
student practice in seeing and completing samples of most of the forms 
encountered as an adult. Twenty-four forms from driver license to 
income tax are explained. This and the consumer education unit would 
be a useful supplement for any adult basic education program. 
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Here are some organizations to which you might write for information 
about self- instructional materials. These are companies whose products we 
have used and would reconmend. 



Behavioral Research Laboratories 
Box 577 

Palo Alto, California 
(programmed materials) 

Craig Educational Products 
921 West Artesia Boulevard 
Compton, California 90220 
213/537-1233 
(reading programs) 

CTB/Mc6raw Hill 
Order Service Center 
Del Monte Research Park 
Monterey, California 93940 

Educational Developmental laboratories. Inc. 
284 East Pulaski Road 
Huntington, New York 11743 
("word clues" vocabulary) 

Educational Marketing Corporation 
5001 West 78th Street, Suite 800 
Minneapolis, Minnesota 

("How to Survive in College," college level study skills) 

Educational Systeras Development 
Box 457 

Royal Oak, Michigan 48068 

E.M.C. Corporation 
180 East Sixth Street 
St. Paul, Minnesota 55101 
612/227-7366 

(spelling improvement series) 

Entelek, Inc. 

42 Pleasant Street 

Newburyport, Massachusetts 01950 

(Vol. I, Elementary-High School Programmed Instruction Guide, $9.50) 

Globe Book Company 
175 Fifth Avenue 
New York, New York 10010 
("Forms in Your Future") 
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Groller Educational Corporation 

845 Third Avenue 

New York, New York 10022 

(consumer education, reading attainment system) 

Harcourt Brace Jovanovlch 
Polk and Geary Streets 
San Francisco, California 94109 
(programmed English series) 

Houghton Mifflin Company 
777 California Avenue 
Palo Alto, California 94304 
("Practical Math for Business") 

Individual Learning Systems 

Post Office Box 3388 

San Rafael, California 94902 

415/479-0177 

(basic skills in English) 

Instructional Objectives Exchange 
Post Office Box 24095, Department A 
Los Angeles, California 90024 

Science Research Associates, Inc. 
259 East Erie Street 
Chicago, Illinois 60611 

Southwestern Publishing Company 
11 Guittard Road 
Burlingame, California 94010 
415/697-7050 

(punctuation, programming approach) 

Teaching Resources, Inc. 
Station Plaza, Bedford Hills 
New York, New York 10507 

Wadsworth Publishing Company, Inc. 

Belmont, California 94002 

(a first program in mathematics, Heywood) 

Westinghouse Learning Corporation 

100 Park Avenue 

New York, New York 10017 

Educational Products Information Exchange Institute 

463 West Street 

New York, New York 11014 

(bulletin and publications about educational products 
Union" of education) 
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EVALUATION PRCJCEOURES AND RESULTS 




TYPES AND PURPOSES OF EVALUATION 
Evaluation has been stereotyped as an activity conducted by a person 
with a Ph.D. in education, who, for a large consultant fee, is invited to 
visit a project. Interview staff, and write an erudite tome about the 
successes and failures of the program. Different concepts were used in 
the evaluation of the Individualized Learning Center at Cook School. 
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BEST ccpif mmii 

W. James Popham {1972: 1), a renowned authority about evaluation has 

said. 

Although there have been minor differences of opinion 
through the years, most educators have conceived of educational 
evaluation as an operation in which the quality of educational 
enterprise Is judged. In other words. for most educators, the 
term 'evaluation' means appraising the worth of an educational 
undertaking such as a curriculum, a course of study, or a parti- 
cular Instructional procedure. Generally, such evaluations are 
undertaken with a view to making declsilons; for example, should 
a course of study be organized in one way or another, should a 
new instruction scheme be adopted, etc. 

As Popham (1972: 3), mentioned and as Is shown In the display 

below, there are two basic roles for evaluation. 



Role 




Purpose 


Role 1 


Needs Assessn%nt 


To determine the desired ends 


Role 2 


Treatment Adequacy 
Assessment 


To judge the worth of the 
educational means 



Judging the worth of an educational neans, according to Popham, can 
occur at different times for different purposes in the project as displayed 
below. 



Time 


Name 


Purpose 


During the Project 


Formulatlve 
Evaluation 


To make Improvements In the 
educational means 


At the Conclusion 
of the Project 


Summative 
Evaluation 


To compavfi the project with 
competitors 
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While the ILC pnjject has con» to the conclusion of Its initial 
funding period, the Instructional system will be continued and revised. 
Therefore, this report Is best understood as a formative evaluation . It is 
Impossible and inappropriate to report, In any final sense, whether we suc- 
ceeded or failed since the intent of this evaluation was to help us make 
improvements in the Instructional systems of the ILC. 

As nffintloned previously in this manual, another dimension of the 
Developmental Learning Laboratory has been to create Instructional products 
which could be used not only In the ILC by Indian and non-Indian educators 
but elsewhere as well. Approximately 1200 of these products have been 
distributed but an Insufficient number of the evaluation forms have been 
returned to be included In this evaluation. Upon return, criterion 
referenced rroasures will be used to judge whether or not the instructional 
objectives in each product have been met based upon student performance. 
This evaluation, therefore, is limited to the instructional systems and 
products used in the Individualized Learning Center at Cook School. 

PROCEDURES 

The population for this evaluation included 24 Indian students 18 
years of age or older who took courses in the ILC one or two semesters during 
the academic year 1973-74. Not included were eight students who took courses 
during the year but who failed for various reasons to take one or more of the 
ILC's operation because we discovered that the Iowa Test of Educational Dev- 
elopment (12th grade version) was Insensitive to student academic performance. 
Evaluations of the first operational year of the ILC are reported in Progress 
Reports one and two. 
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Population Characteristics 

The ages of the 24 students range from 18 to 50. Seventy-five 
percent of the students were high school graduates before they began In- 
struction. The median grade completed was tith grade. In regard to mar- 
ital status, 58^ of the evaluation population were married. Some of the 
students were taking 6 or more hours of college while taking courses in the 
ILC. Twenty-eight of the students took 6 or more hours of college work at 
Mesa Community College or Arizona State University during the fall and/or 
spring semester of 1973-74. Seven tribes were represented In the population 
with the largest representations from the Episcopalian, Presbyterian, and 
Christian Reformed churches. 

Program Characteristics 

During the fall semester 1973, students in the ILC contracted for 350 
hours of credit and completed 322 hours or 92^ of their contracted credit 
hours. During the spring sanester 1974, students contracted for 226 hours 
of credit and completed 187 hours or 83% of their contracted credit hours. 

On the average during the fall semester, students took 14 hours of 
their study program in the ILC and during the spring sanester, took 11 hours. 
During the fall semester 3,947 hours of instruction was scheduled and stu- 
dents completed 3,746 or 95%. During the spring semester 1974, students 
scheduled 2,428 hours of instruction and completed 2,380 or 98.1%. When 
attendance records were coirtbined for the fall and spring semesters, the 
average student attended 99% of the hours that he scheduled. For more 
information about program characteristics, see Tables I and 11 In Appendix A. 
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Design 

For administrative reasons It was necessary to use a quasi ^experimental 
(one group only) design. We thought It unwise to use an experimental and con- 
trol group In the evaluation because It would have meant withholding programs 
from students for at least a year's period of time. Using an experimental 
and control group also would have reduced the size of the evaluation popula- 
tions. 

Instruments 

In this evaluation we sought to measure in the cognitive and affective 
domains. Three Instruments were used. 

1) The California Achievement Test to measure academic achievement in 
each of the basic studiy areas provided In the ILC» (Cognitive) 

2) The Value Orientation Scale to measure changes in student cultural 
values, (Affective) and 

3) a Student Attitude Inventory constructed by ILC staff to measure 
the attitudes of students toward various components of the ItC. (Affective) 

The California Achievement Test, published by CTB/McGraw Hill, has 
three academic levels and two compatible versions which measure in the 
following areas: reading vocabulary, reading comprehension, mathenatics 
confutation, mathematics problem solving, language arts usage, language arts 
mechanics, and spelling. The California Achievement Test (referred hence- 
forth as the CAT) yields scores in grade equivalency. In other words 
using the CAT it is possible to measure the year and the month before a 
student begins instruction and the year and month at the conclusion of his 
studies. One of the reasons that we selected the CAT was that it yielded a 
unit of measure (grade equivalency) which is readily understood by students 
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and others. During the first year of the ILC project, the Iowa Test of 
Educational Developinent {12th grade measure) was used, but other measures 
convinced us that many students were making academic gains but were not 
coming up to the beginning level of measurement of the ITED. For docwnen- 
tatlon about this point, see Progress Report No. 1. 

To determine which level of the CAT to take, each student completed a 
short "locator test," The locator test is part of the Adult Basic Education 
program adapted from the CAT and published by CTB/Mc6raw Hill. 

Each student taking courses in the ILC completed the CAT before they 
began instruction (pre- test) and after they completed one or two semesters 
(post-test). Because of the traditional problem of student negative attitudes 
at the end of the school year, the CAT post- test given in May 1974 was 
administered in an unusual way. Staff of the ILC requested that a "special 
day" be created for actainistering the post-test. Staff held a breakfast for 
all students to insure that they were awake, well fed, and relaxed before 
taking the test. Students were also rewarded by being allowed to skip the 
rest of their classes after the test. A casual survey of students completing 
the CAT post- test convinced us that students "did their best." 

The Value Orientation Scale was also administered as a pre- and post-test. 
The Scale, developed by Fred L. Strodtbeck,is a measurement of acculturation 
to middle-class values. The VOS consists of eight true and false questions. 
A copy of this Instrument appears in Appendix B. The first three questions 
in the VOS deal with time orientation and mastery over one's destiny. Questions 
four to six measure famllism versus individualism and loyalty to extended 
family versus loyalty to nuclear family. Question seven tests for individual 
versus group orientation. Question eight deals with immediate versus post- 
poned gratification. In a study conducted by the Southwest Cooperative Regiona 
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Laboratory entitled Analysis of Academic Achievement of Indian School 
Students in Federal and Public Schools . It is discovered that the relation- 
ship between student value orientation and academic achievment as measured 
by grade point average was statistically significant although the strength of 
the relationship was rather weak (correlation coefficient of .359.) In other 
words, the degree to which Indian students held raiddle-class values rather 
than traditional Indian values had sanething to do with their acad^ic 
achievement. 

A Student Attitude Inventory was developed by IIC staff at the end of 
the first year of operation. The Inventoiry included, initially, 26 items. 
Using a five point Likert-type Scale, students were asked the degree to 
which they agreed or disagreed with statements. A copy of this inventory 
can be found in Appendix 6. 

Reliability and validity coefficients of the California Achievement Test 
in this evaluation were those stated by the publisher. To the best of our 
knowledge no coefficients have been computed for the Value Orientation Scale 
nor did we compute them for the Student Attitude Inventory. 

Evaluation Questions 

Because this manual is being written for persons with limited or no 
background in statistics and research methods, we shall state the questions 
we considered rather than the usual hypotheses. Tables and other findings 
important to researchers have been placed in the appendices. 

As we evaluated the ItC we sought answers to questions in order to make 
improvements in our learning systems. 
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2. yhat c^cvi students nciJ do aa a result of studying in the TIC? Thts 
question Is ar»other way of asking what criterion levels have students reached in 
what instructional packages. The answer to this question tells the specific 
skills which students gained as a result of their studies. 

2. Did students taking aoursee in the ILC either one oi} two semeatepa make 
statistically sigtiifiaant gains in the eoupsea which they took as measured by 
the subsaales of the California Achievement Test? By kn«*ing the answer tO 
this question we felt that we could determine the effectiveness of the ILC's 
instructional program. 

0, Is it possible to predict uhat students ccm be expected to gain aoademi- 
catty in each of their study areas as measured by the subscate of the California 
Achievement Test? If this question could be answered, we would have helpful 
information for student guidance and for revising our instructional systems. 
For example, it would be unfair to compare on'* student who gains two years in 
reading with another who gains only one year if their academic backgrounds are 
different. Students want to know if they are gaining what they "should," but 
how does one determine how much a student can reasonably gain when his background 
and other factors are taken into consideration? 

4. Do students taking courses in the ILC one or U^o semesters make any 
statistically significant changes in their cultural value orientation as 
measured by the Value Orientation Scale? By knowing the answer to this question 
we can determine the adequacy of a popular theory that Indian students must 
acquire non-Indian or middle-class values in order to succeed In school. 
Individualized instruction, with Its accompanying technology, on the surface 
appears to be non- Indian. Would students have to change their values in order 
to succeed in the ILC? It was our hope when the ILC was started that students 
could maintain their cultural values and yet make academic gains. 
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5. Hoii do students taking courses in the ILC one two semesters 
feel about the various aspeaU of the Learning Center as measured by a 
staff developed Student Attitude Inventory? By knowing the ansnffir to this 
question we will be able to make adjustments based upon the direct opinions 
of students as well as their performance measured by other instruments. 

6. How did the attitudes of students tojard the IZC at the end of 
the second year compare with the attitudes at the end of the first year? 
In other words, did the "newness" wear off and did students become less 
satisfied with the ILC? 

7. Is it passible to identify hiographioal and academic factors 
which contribute to student acadeaio achievement? By knowing the answer to 
this question we can identify which programs work for what types of students 
under what sort of programs. This information *«>uld be helpful in nuking 
revisions in instructional programs and also helpful in guidance counseling. 
If we discover, for example, that the age of the student when he begins work 
in the ILC makes a difference to his academic performance, then it becomes 
necessary to either screen out certain students on the basis of their age 
or mai:«j revisions in the instructional systems. 

Results 

The results of this evaluation will be reported in answer to each of 
the evaluation questions. To answer the first 6 questions, all statistical 
analyses were done using the ILC's Monroe Model 1766 Programmed Calculator 
and statistical packages furnished by the Monroe Company. To answer the 
7th question, it was necessary to use the computer facilities of Arizona 
State University under a contractual arrangement. 



80 



2. What aan students now do as a result of studying in the ILC? 

In LANGUAGE ARVS, all of the students can now write a topic senttnce, 
a paragraph, a thesis statement, and a exposition. Nearly all of the stu- 
dents can Identify the eight parts of speech, can capitalize correctly, and 
punctuate accurately* All of the students have learned the meaning of at 
least 150 new words and can spell these words correctly and half of the 
students have learned from between 200-300 words and can spell them correctly. 

In SPEED READING all of the students are now reading at least 260 words 
per minute with 35% comprehension or better. Several of the students are 
reading above 450 words per minute with comprehension about 95%. One 
student Is now reading 512 words per minute with 100^ comprehension. 

In MATHEMATICS all of the students can now add, subtract, multiply, and 
divide whole numbers. Most of the students can add, subtract, multiply, and 
divide fraction and several students are able to do algebraic formulas and 
can solve equations. 

In STUDY SKILLS, nearly all of the students now know how to use the 
library card catalog, the Reader's Guide to Periodicals , an encyclopedia, 
the World Almanac , and a dictionary. All of the students now know how to 
use books Including the ability to use the table of contents, the Index, 
glossary, suggested readings, and illustrations. In regard to test taking, 
all students now can Identify and name types of tests, can make the appro- 
priate responses for different types of tests, can list four general rules 
In test taking, can name and identify types of grading systems, and can 
describe the meaning of "grade equivalent," "percentile rank" and "percentage 
of the total." 

Other courses. In addition to basic studies, were offered In the ILC. 
These courses Included Biblical, theological, and cultural studies. Similar 
statements to those made about abilities in basic studies can be furnished 
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about other courses upon request. These types of n»asures (statenients) 
are called criterion- referenced measures irtiich n%ans that certain numbers 
of students mt certain criteria or levels In the various Instructional 
packages so that students new* have skills which thesy did not have prior to 
their studies. To report more specifically In the use of criterion- 
referenced measures would require more space than Is warranted In this 
manual. Such information, however, is available for those who are serious 
about establishing learning centers. 

2. Did etudente leaking aemraes in the ILC either one or two eematere 
make statistically eignifiomt gains in the ooursee which they took as 
measured by the Bubsoales of the California Achievement Test? 

The general answer to this question was yes; students did make statis- 
tically significant gains In reading vocabulary, reading conprehension, 
mathematics computation, mathematics problen solving, language arts usage, 
and language arts mechanics. Students did not make statistically significant 
gains in spelling although we expect this since no Instruction was provided 
which was directly related to the task of learning spelling. 

To arrive at these answers a t Test for Paired Observations was run. 
With 95% confidence we can say that the gains which students made were actual 
rather than a matter of chance. The following display shows where on the 
average students started before Instruction, where they finished after par- 
ticipating in instruction, and the average amount of student academic 
achievement. 
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Academic Achievement Measured by California Achievement Test* 



Academic Average Academic 
Area Level Before 

Instruction 


Average Academic 
Level After 

Instruction 


Average Amount 
of Gain 


Reading Vocabulary 


7.2 




+1 .2 


Reading Comprehension 


7.7 


8 7 


+1.0 


Reading Total 


7.4 


8 8 


+1.0 


Math. Computation 


6.3 


7 g 


+1.7 


Math. Problem Solving 


5.7 


6 6 


.8 


Math. Total 


6.0 


7.4 


+1.4 


Language Arts Mechanics 


8.5 


9.8 


+1.3 


Language Arts Usage 


6.9 


8.5 


+1.6 


Language Arts Total 


8.0 


9.3 


+1.3 


Spelling 


10.2 


10.9 


.7 



♦Levels are stated in years and months. Example: 7.1 means 7th year, 1st month. 
The range of student gains was very large. Some students gained as much as 

five years in two semesters and several others lost as much as two years. The 

following display shows the range of gain or loss for each of the subscale areas. 



FOR SPJDFNTS TAKING COURSES ONLY ONE SEMESTER 



Reading Vocabulary 
Reading Comprehension 
Mathanatics Problem Solving 
Language Arts Mechanics 
Language Arts Usage 
Spelling 



Lowest Gain Score 
-.4 

+1.0 

-1.4 

+ .1 

-1.5 

-1.9 



Highest Gain Score 
+2.3 

+2.7 

+3.2 

+3.2 

+2.4 

+4.1 



ERIC 



83 



FOR STUDENTS TAKING COURSES THO SEMESTERS 



Reading vocabulary 


loi«est Gain score 
-1.1 


Highest Gain Score 


Keduins voraprenens ion 






Mathematics Computatfon 


- .4 


+5.1 


Mathematics Problem Solving 


- .9 


+2.6 


language Arts Mechanics 


-1.7 


+5.2 


language Arts Usage 


-1.0 


+4.7 


Spelling 


-3.3 


♦4.4 



For more statistical Information about this analysis, see Table III In the 
Appendix. 



3. le it poseibte to predict what atudmta tfwt be eaqpeated to gcein 
GLctademioatty in each of their etudy areas as maaurea by the sttbst^tes of the 
California Aohievemnt Test? 

The general answer to this question was that Vfe could predict In some sub- 
scales but not for others. 

To find the answer to this question, we had to first calculate what students 
could be expected to gain using the following formula: 

C.A.T. Pre-test score 

» Expected Gain 

Grade completed + 1 
The Bureau of Indian Affairs used this formula In their evaluation of 
Title I programs in the state of Arizona. In the formula an attempt Is made to 
account for the educational backgrounds of students as measured by how many 
years they completed In school before beginning instruction and their educational 
entry level as measured by their pre-test score on the California Achievement Test. 
While there are more accurate ways of predicting student academic achievement^ 
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we felt that this formula should be analyzed because of Its simplicity. What 
we hoped to predict was the minimum that a student could be expected to gain 
in each subscale of the CAT rather than the maximum or actual amount of ex- 
pected gain. 

To answer this question, we used a t Test for Paired Observation utilizing 
the expected gains and actual gains in each subscale area of the CAT for students 
taking courses in the ILC one or two sanesters. With 95% confidence we can 
state that there wa^ statistically significant differences between expected and 
actual gains in the following areas: 

Reading vocabulary 

Mathematics computation 

Language arts usage 

There were not statistically significant differences in the following areas: 

Reading comprehension 

Mathematics problem solving 

Language arts mechanics 

Spelling 

Another analysis was done to compute the percentage of students who met 
or exceeded their expected gains in each subscale area. 

CAT Subscale Area 



ERIC 



Percentage of students 





expected gain 


Reading Vocabulary 


76% 


Reading Comprehension 


78% 


Mathematics Computation 


88% 


Mathematics Problem Solving 


63% 


Language Arts Mechanics 


57% 


Language Arts Usage 


71% 


Spelling 


38% 



c:?T r.9^ available 

4. Do students taking oonrses in the ILC one or tuo s&nestera make any 
statistically significant changes in their cultural values orientation as 
measured by the Value Orientation Scale? 

The general answer to this question was no. When a t.Test for Paired 
Observation was run using the pre- and post-tests of the Value Orientation 
Scale, we found that while there was a numerical shift the difference was 
not sUtistically significant. We can state this with 95% confidence. 

The displays below show where students on the average started and 
finished in their value orientation on a continuum from traditional Indian 
values to non-Indian or middle-class values as measured by the Value Orienta- 
tion Scale. It should be remembered that while to the naked eye It appears 
as though there was a shift the difference could be due to chance in measure- 
ment. 

CULTURAL VALUE ORIENTATION BEFORE INSTRUCTION 

White 

-aditlonal I- Before Middle- 

VaiieJ «♦ 6 B2ii ^ 6 832 tt 5 842 »♦ e 852 k b' 



After 

Instruction 



I Before ...Mw.« 

r" Inst ruc tion Class 



Tprtrrrs values 



It is of interest that eight of the students moved from a preference for 
Irnmediate gratification and toward a preference for delayed gratification. 
Eleven of the students, however, made ro change along this continuum. The 
analysis showed that this change was not statistically significant. For more 
data about this analysis, see Table IV in the Appendix. 

5. Hou dr> students taking aouFses in the ILC one or tufo semesters feel 
about various aspe -^ts of the Learning Center as measured by a staff developed 
Student Attitude Inventory? 
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1. When students were asked If they felt they learned more about a 
subject in a regular classroom than In the Individualized Learning Center , 
53^ agreed or strongly agreed while 41% disagreed. 

2. Eighty-eight percent of the students agreed or strongly agreed that 
becoming responsible for their own learning is as Important as getting to know 
a particular subject. 

3. When given the statement "Most students work harder and get more done 
In an Individualized Learning Center than in a regular classroom," 59% of the 
students agreed or strongly agreed, 24^ were unsure, and 17% disagreed. 

4. Fifty-six percent of the students strongly agreed or agreed that the 
objectives of Instruction are more clearly stated In the courses In the ILC 
than in most of their other classes, and 24% of the students disagreed. 

5. When asked if the immediate feedback (knowledge of results) provided 
by the materials in the ILC made learning easier, 82% of the students strongly 
agreed or agreed. The remaining students were unsure. 

6. Students were asked how they felt about knowing how well they were 
doing in their courses in the ILC, One hundred percent of the students agreed 
or strongly agreed that it is Important for them to know. 

7. Forty-seven percent of the students agreed or strongly agreed that 
they learned more from a cassette tape than from a lecture in a classroom and 
35% of the students were unsure. Eighteen percent of the students apparently 
felt that they learned more from a lecture in a classroom. 

8. When asked if they felt that it is easier to learn from programmed 
books than from a regular textbook, 57% of the students strongly agreed or agreed 
and 35% of the students were unsure. 

9. Eighty- three percent of the students disagreed or strongly disagreed that 
programmed materials are confusing to them. 




87 



10. "I work harder when I am competing with other students than when I 
am working on own." Reactions to this statement were mixed. Fifty-three 
percent of the students agreed or strongly agreed and 41% of the students 
disagreed or strongly disagreed. The remaining students were unsure. 

11. When asked if the teacher 2 tn the ILC were usually able to give the 
help needed, 88% of the students agreed or strongly agreed and 12% disagreed 
or strongly disagreed. 

12. Only 12% of the students agreed or strongly agreed with the statement 
that teachers in the ILC did not seem to be interested in how well they were 
doing. 

13. When asked if students felt that they had a more poslt.ve attitude 
toward learning since they had been in the ILC, 70% of the students agreed or 
strongly agreed while 24% of the students were unsure, and 6% disagreed. 

14. Ninety-five percent of the students strongly agreed or agreed that 
the materials in the ILC are Interesting. 

15. The students were asked If working in the ILC at their oi^n rate is 
of importance to them. Eighty-eight percent agreed or strongly agreed. 

16. Students were asked If they felt It Important to work privc tely so 
that others don't know when they make mistakes. Surprisingly, only '3B% of the 
students agreed or strongly agreed while 41% of the students disagreed or strongly 
disagreed. The remaining students were unsure. 

17. Did the ILC make students more responsible for their own learning? 
When asked about this matter, 83% of the students agreed or strongly agreed that 
the ILC did help them become more responsible for their own learning while 12% 
disagreed. 

18. Seventy-six percent of the students agreed or strongly agreed that It is 
Important that grades for courses in the ILC not be dependent upon what other 
students do. 
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19. The Student Attitude Inventory Included a request that students list 
their opinions about the courses that they felt should be taught as a group 
class Instead of in the ILC. Two students mentioned English, three students 
mentioned Bible courses, 1 student mentioned study skills, 1 student mentioned 
roathenat1cs,and the remaining students had no opinion. 

20. The following items were listed by students as those factors which they 
disliked about working in the ILC: It is noisy (3 students), it is too quiet 

(1 student), it is too hot (1 student). 

21. When asked what students felt they liked best about working in the 
Individualized Learning Center, the following matters were mentioned: 

A. "Reading and basic English" 

B. "You have your teacher, tapes, and materials." 

C. •'We are able to make up classes when we miss them." 

D. "The individual work that we do" 

E. "You can concentrate." 

F. "Being alone and just sitting there and nobody knowing if you are 

working or not" 

G. "Working at rny own rate: "(mentioned by 6 students) 

22. To make the Individualized Learning Center a better place to learn, the 
following suggestions were made by students: 

A. "Do not permit visitors in the ILC." 

B. "Bring in current learning devices to use and more of the latest 
materials on learning." 

C. "Expository writing in college" 

D. "Instructors to meet with students at times to discuss progress. 
A majority of students did not have a response to this request for 

suggestions. 
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6» Hou did the attitudes of studentB ioward ike ILC at tiie end of the 
eeoond year aompare iHth the attitudes at the end of vhe first year? 

When the responses to items for the second year were compared with the 
responses from the first year. It was found. that no statistically significant 
differences existed for each Itan EXCEPT one. More students this year than 
last year felt that they could get more done In a regular classroom than In 
the ILC. To make these comparisons a Chi Square (2x3) Analysis was done. The 
five cells were collapsed Into three. With 95% confidence we can say that the 
results are statistically significant rather than a matter of chance. 

?, Is it possible to identify biographical and aoademio factors whioh 
contribute to student aoademio achievement? 

This question actually contains two sub-questions; 1) How well can we 
predict student academic performance, and 2) What factors are contributing to 
academic perfonmince in what amounts? 

To answer this question Multiple Regression Analysis was done on a UNIVAC 
computer at Arizona State University. What was discovered from the analyses is 
that it is possible to predict student academic performance very well in six of 
the eight C.A.T. subscale areas (R =» above .84). 

This type of knowledge of how students will do in the ILC before they begin 
instruction based upon Information collected prior to academic work is Important 
because it helps to determine what programs a student should use, under what 
conditions, and what reasonable results to expect. The ability to predict 
also helps to make modifications in the Instructional program by knowing what 
worked for whom. 

Where it was possible to predict student C.A.T. post-test scores, the 
following factors were identified as contributing significantly to student 
academic performance. These factors are listed in the order of how much they 
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contribute to student performance. 

1.0 READING VOCABULARY (R^ « .85) 

1.1 C.A.T. pre-test 

1.2 Age 

1.3 Marital status 

2 

2.0 READING COMPREHENSION (R = .94) 

2.1 C.A.T. pre-test 

2.2 Number of semesters in related courses 

3.0 READING TOTAL (R^ = .94) 
3v1 C.A.T. pre-test 

3.2 Number of semesters In related courses 

4.0 MATHEMATICS COMPUTATION (R^ = .87) 

4.1 C.A.T. pre-test 

4.2 Grade in school completed before starting instruction 

5.0 MATHEMATICS PROBLEM SOLVING (R^ = .85) 

5.1 C.A.T. pre-test 

5.2 Number of semesters in related courses 

6.0 Mathematics Total (R^ = .91) 

6.1 C.A.T. pre-test 

6.2 Grade in school before starting instruction 

It was not possible to predict student academic performance as well in 
the areas of language arts. In language arts, however, the factors contributed 
to how students performed acadenically. 

7.0 LANGUAGE ARTS MECHANICS (R^ « .75) 

7.1 C.A.T. pre-test 

7.2 Number of semesters in related courses 

7.3 Marital status 

8.0 LANGUAGE ARTS USAGE (R^ = .75) 

8.1 C.A.T. pre-test 

8.2 Value Orientation Scale score 

8.3 Age at the time of beginning Instruction 

9.0 LANGUAGE ARTS TOTAL (R^ = .77) 

9.1 C.A.T. pre-test 

9.2 Number of semesters in related courses 
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CONaUSIONS AND RECOM^£NDATIONS 

1 . STUDENTS TAKING COURSES IN THE ILC ONE OR TWO SEMESTERS DURING ACADEMIC 
YEAR 1973-74 MADE ACHIEVEMENT GAINS THAT MERE STATISTICALLY SIGNIFICANT IN THE 
BASIC STUDIES AREAS OF READING VOCABULARY. READING CffllPREHENSION, MATHEMATICS 
CCWPUTATION, MATHEMATICS PROBLEM SOLVING, LANGUAGE ARTS USAGE AND MECHANICS, BUT 
NOT IN SPELLING. 

Because of this fact it is reasonable to infer that the instructional 
packages in the ILC and the educational systems are affecting student learning as 

measured by the CAT. 

Students did not make gains in spelling that were statistically significant, 
but this was expected because no instructional programs in spelling were offered. 
Only four of the students were spelling below the eighth grade level in the fall, 
and only five students were spelling below the established program criteria 
level of eighth grade in the spring. The average level of spelling ability before 
instruction was 10 years and 2 months, and after instruction it was 10 years and 
8 mon^s; therefore, spelling was not a major problem area. 

How students made gains is of equal important to tatfiat was gained. Prior to 
the establisNnent of the ILC, courses in basic studies were offered in the trad- 
itional manner. Group classes were held at set timts with all students studying 
the same materials regardless of their educational entry level. When students 
missed class due to illness and when students had to return home because of 
family concerns, a "make up" situation was created which became difficult for 

student and teacher. 

The ILC has, therefore, solved some administrative problens. It is now 
possible for students to study at their own pace, on their own level, and at a 
convenient time. The necessity of scheduling the right class for the right 
student at the right time has been eliminated. 
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While It Is too early to know for sure. It appears that studies in the 
ILC reduce the amount of time required to prepare for the G.E.D. exam. In pre- 
vious years it took students at least one year (Z semesters) to pass the G.E.D. 
test. In the IIC (with only one exception) the students coiiq)1eted the G.E.D. 
test in one semester or less. 

Individualized instruction has also helped to solve a faculty-student 
ratio problai. In the traditional approach to education it is necessary to 
have fairly large classes to justify teaching staff. The ILC approach makes it 
possible and econffinically feasible to offer a number of courses where student- 
faculty ratios are as low as 2 to 1 because it is possible to add courses with- 
out adding staff. 

Because of the ILC, student academic performance as measured by grade point 
average has also been improved. As reported earlier, the schooVs grade point 
average was raised cne entire letter grade from a C average to a 6 average. 

The so-called "attendance problem" has also been apparently solved in 
the lie. Where prior to the ILC students on the average had as high as a 40% 
absence rate, the overall absence average in the ILC was less than 2%. The 
possibility of making up work that had been missed accounted for the improved rate 
of attendance. 

In the typical approach to education it is difficult, if not impossible, 
for students to make up work. In the ILC when students must return home for 
funerals and other family concerns, it is possible to "freeze" or stop a student's 
program. In most classes if a student misses two or three weeks of work. It be- 
coRffis necessary to drop the class. In the ILC students can pick up where they 
left off without being penalized for absence. 

Students make statistically significant gains. That is true but it Is 
also Important that these gains were made in a way that (at least for us) tsakes 
the ILC an attractive alternative to the traditional approach to education. 
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2. STUDENTS ON THE AVERAGE DID BETTER IN SOME BASIC SKILLS AREAS THAN 
IN OTHERS. 

Students made higher* gains in the CAT subscale areas of reading vocabulary, 
mathematics computation and language arts mechanics than they did in reading 
comprehension, mathematics problem solving, and language arts usage. Why? 

While this is only a hypothesis, it appears that students did better 
academically in areas requiring "lower order" mental activities. The term 
"lower order" refers to the beginning levels of a cognitive taxonoiry. For 
example, in courses like reading comprehension where students need to synthesize 
and evaluate they did not do as well as in courses like reading vocabulary where 
. memorization and recall are the primary skills required of students. In spelling 
where learning is primarily a matter of memorization, students did very well 
(an entry level of lOth grade or better for the average student). 

If the hypothesis is true that students can function better at lower order 
cognitive levels, it could be due to poor educational backgrounds or to cultural 
factors. The answer to why students did better in some areas than in others can 
not be determined from the data. It is reasonable, however, to consider that a 
cultural dimension exists in relation to the use of various cognitive skills. 
For example, to be able to "evaluate" one must believe that it is legitimate to 
doubt and question. Students educated in a paternalistic educational system 
may have failed to develop the necessary attitudes prerequisite to the acquisi- 
tion of certain higher order cognitive skills. Additional investigation is 
needed in this area. 

3. WHEN THE FORMULA TO CALCULATE STUDENT EXPECTED ACADEMIC GAINS WAS USED. 
IT WAS POSSIBLE TO FAIRLY WELL PREDICT STUDENT PERFORMANCE IN THE AREAS OF 
READING VOCABULARY, MATHEMATICS COMPUTATION. AND LANGUAGE ARTS USAGE BUT NOT IN 
THE AREAS OF READING COMPREHENSION. MATHEMATICS PROBLEM SOLVING, LANGUAGE ARTS 

MECHANICS. AND SPELLING. 
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As mentioned earlier, the t Test for Paired Observations using actual 
and expected gains was more for the purpose of examining the formula than 
for considering student achievement. 

When one considers that from 70 to 80Z of the students exceeded their 
expected gains in the various subscale areas of the CAT, the formula looks 
"good," but is the formula accurate? 

In "snooping the data," it was found that the formula consistently 
failed to estimate the minimal level of expected gains for students in the 
middle range of entry scores. In other words, the formula calculates expected 
gains better for students with poor or excellent educational backgrounds as 
measured by the CAT pre-test. 

What is wrong with the formula? A partial answer is given to this 
question by the results of the Multiple Regression Analysis which show that 
other factors not considered in the formula account for student performance 
(at least at Cook School). 

Another flaw can be seen in the formula itself. The CAT pre-test scores 
are calculated on a ten-month year, but in the formula a year is measured by 
a (1) for each year of schooling that the student has completed before start- 
ing instruction. 

4. STUDENTS IN THE ILC DID NOT MAKE ANY CHANGES IN THEIR CULTURAL VALUES 
ORIENTATION WHICH WERE STATISTICALLY SIGNIFICANT. 

If the Vaiue Orientation Scale measures what it is intended to measure 
(the value orientation on a continuum from traditional Indian values to middle 
class values), we can conclude that studies in the ILC do not make students 
"white men." This conclusion is further supported by an analysis in which the 
relation between value orientation and academic performance was considered 
(correlation between V.O.S. pre- and post-tests with achievement gains). The 
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relation between value orientation and achievement gains was very, very weak. 
In other words, it DID NOT HOLD TRUE that the more middle class a student was, 
the better he or she did academically. The popular theory that an Indian 
student has to be WHITE TO BE RIGHT in school did not hold true in the ILC. 

5. ON THE BASIS OF THE RESULTS FROM THE STUDENT ATTITUDE INVENTORY IT 
APPEARS THAT STUDENTS GENERALLY FEEL POSITIVE TOWARD THE ILC. 

A large majority of the students felt positive toward the various com- 
ponents of the ILC system Including the use of instructional objectives, fre- 
quent feedback of results, individualized help, programmed texts, and audio- 
visual materials. 

From the results, however, it can be concluded that a few students would 
prefer to have a choice of either studying in the ILC or a traditional type 
of classroom. This conclusion is based upon the results of student responses 
to inventory items seven, eight, and ten in which a few students expressed 
an interest In hearing lectures rather than using tapes and in competing with 
other students. 

It was puzzling to discover that a large percentage of students have less 
interest in "privacy" in learning than staff has assumed. The so-called 
"embarrassment factor" which was important for staff to prevent (embarassment 
about failinq) was of less importance to students than teachers. 

Of heartwarming significance to ILC staff was the fact that 70% of the 
students indicated that they now feel more positive about learning after having 
been in the ILC than they did before they started. If we have in fact "changed 
attitudes" about education, then we may have achieved something which cannot 
be measured for years to come. 

The fact that the results of the Student AttUude Inventory at the end of 
the second year of the ILC were not significantly different from the results at 
the end of the first year indicates that student feelings about the ILC are not 
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affected by the length of time that they spent in their studies in the ILC. 
In other words, students have not (as yet) grown tired of the ILC. 

6. WE CAN PREDICT STUDENT CAT POST-TEST SCORES VERY WELL IN THE AREAS 
OF READING VOCABULARY, COMPREHENSION, AND MATHEMATICS COMPUTATION AND PROBLEM 
SOLVING BUT NOT WELL ENOUGH TO MAKE MODIFICATIONS IN THE INSTRUCTIONAL SYSTEMS 
AS YET. 

The purpose of "accurate prediction" of student academic peformance before 
instruction by knowing certain characteristics of students is to determine what 
programs will work for what students under what conditions with what expected 
results. 

CAT pre-test scores and the number of semesters spent in related courses 
consistently had the strongest relation with student performance (CAT post-test 
scores) in most subscale areas. 

Perhaps this conclusion is clearer by considering the negative side of 
the results. Our analyses show that more often than not the age of the student, 
the highest grade completed before starting in the ILC, marital status, and 
amount of time spent in study HAVE LITTLE IF ANYTHING TO DO WITH how well a 
student does academically. Does this mean that we have truiy "individual i7ed" 
or that other factors which we did not measure make the significant difference 
in how well students perform academically? 

It should be noted, however, that in Reading Vocabulary AGE AND MARITAL 
STATUS DID MAKE A DIFFERENCE (contribute to the percentage of variance). In 
Reading Comprehension, Mathematics Computation and Problem Solving the nuinhfr 
of semesters which students spent in related courses DID CONTPIBUTI t.o -.tuflfrit. 
performance. 

Several recommendations '".hould bo considered based upon t.hf.o r<.".uli'.. 
First, the use of Multiple Regression Analysis should be contimif-'J Mri'.- ihr 
results may be due to sample size ds much as to actual effe(.t'>. If r. nc •••.•..)! y 
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and important to know what factors are affecting student learning In order to 
control these factors In the students' Interests. 

Second, and this recoiranendation seems obvious. If a student can choose 
between studying for one or two semesters In a given area, he or she should 
study for two semesters because he or she likely will do better. This 
recommendation does NOT hold true, however, for Reading Vocabulary and 
Language Arts Usage where the number of semesters spent in study have little 
if anything to do with student performance. 

Third, the continuation of some students studying for as much as 160 hours 
in some subject areas for a semester should be stopped because of the lack of 
justification for this effort. We have known for a long time that there Is a 
point of "diminishing returns" in learning, and this holds true In the ILC. As 
yet we do not know what students under what conditions should study for what 
length of time. It is important to know this because each student's contracts 
for what will be studied and for how long. Unfortunately, guess work will 
have to be continued. 

Concluding Remarks 

The purpose of this evaluation was to identify areas for improvement in 
programs (formative evaluation). We have not proven in any definitive way 
whether the ILC is a success or failure. The analyses show that many of the 
procedures in the ILC should be continued because they are affecting learning 
and because student attitudes are positive about these method?. Considerable 
work lies ahead. It will be important to identify which specific instructional 
packages WITHIN academic areas work best for what types of students under what 
conditions. Since it is necessary to buy cofwiercial packages without knowing 
how well the materials will work, careful evaluation is important to measure 
effectiveness and cost before additional funds are expended. 
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When Cook School submitted its proposal to Lilly Endowment, Inc., we 
asked for a chance to prove that the ILC approach would work. We provided 
reasonable assurance as the Pima Indians would say that "YH THAW EA JU" (it 
would happen). We feel that the ILC hasi 
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APPENDIX A 
DEMOGRAPHIC FACTORS 
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TABLE III 



Percentage of Students Exceeding Expected Gains 
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Exceeded expected gain 
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Did not exceed expected gain 
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Total number of students 
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Lowest gain score 


-.4 


+1.0 




-1.4 


+ .1 
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-1.9 


Highest gain score 
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42.7 
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+2.4 


+4.1 


IN COURSE WO SEMESTER 
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7 


9 


5 


Did not exceed expected gain 
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Total number of students 
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14 


14 


14 


Lowest gain score 


•1.1 


-2.0 


-.4 


-.9 


-1.7 


-1.0 


-3.3 


Highest gaii score 


-3.4 


+3.3 


+5.1 


+2.6 


+5.2 


+4.7 


+4.4 


COMBINATION OF ONE AM) TOO SEMESTERS 
















Exceeded expected gain 
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15 
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Did not exceed expected gain 
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13 


Total number of students 


17 


17 
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16 


21 
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Percent exceeding expected gain 


76y„ 


787o 
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637o 


577o 


717o 


387. 
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TABLE IV 
Changes in Value Orientations 





I'ro-Ti'st 


Post Test 


Number 


20 


20 


Mean 


6.10 


5.65 


Standard Deviation 


1,25 


1.35 



Mean Difference; 


.45 


t 


+ .169 



Not significant at the .05 level. 
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TABLE V 



MULTIPLE REGRESSION EQUATIONS TO PREDICT C.A.T. POST TEST SCORES 

1. Reading vocabular-y - .v83 (C.A.T. pre-test score) + .147 (age) + 

-.841 (marital status) + -.183 

2. Readinq comprehension = .926 (C.A.T. pre-test score) + -3.163 

(no. of semesters in related courses) + 
6.038 

3. Reading battery total = .745 (C.A.T. pre-test score) + -2.00 

(no. of semesters in related courses) + 
1.987 (expected gain score) + 4.718 

4. Mathematics computation = 1.331 (C.A.T. pre-test score) + .294 

(grade completed) + -3.345 

5. Mathematics problem solving - 1.222 (C.A.T. pre-test score) + 

-1.056 (no. of semesters in 
related courses) + 1.177 

6. Mathematics battery total = 1.324 (C.A.T. pre-test score) + .218 

(grade completed) + -5.527 (no. of 
semesters In related courses) + -1.926 

7. Lanauage arts mechanics = .886 (C.A.T. pre-test score) + -.806 (no. 

of semesters in related courses) + .787 
(marital status, married = 2, single = 1) 
+ 2.208 

8. Lanquagp arts usage = 1.172 ^ 1.172 (C.A.T. pre-test score) + .966 

(V.O.S. pre-test score) + -.141 (present age) 
+ -1.790 

9. Lanquane arts battery total = 1.014 (C.A.T. pre-test score) + .625 

(no. of semesters in related courses) + 2.215 
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TABLK VI 

Academic Aciii tvomont Moasurtd by California Achievement Tost 
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All areas sir.ni f icant ai the .05 Ifvi-l or hi};hc'r except Spcllinr,. 
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TABLK VII 

Comparison of California Achievement Test Actual and i:xp«cteti Cains 





Expected Gains 


Actual Gains 


Mean 
Diff. 


t 

Value 




N 


Mean 


St. I). 


N 


Mean 


St . D. 






Reading Vocabulary 


17 


0.45 


0.19 


17 


1.10 


1.20 


+ .65 


+2.25 


Reading Oompreliension 


17 


0.51 


0.22 


17 


0.87 


1.58 


.36 


+0.94 


latM Computation 


16 


0.47 


0.22 


16 


1.43 


1.24 


+ .96 


+4.01 


Math I'roblem Solving 


16 


0.42 


0.20 


16 


1.07 


1.35 


+.65 


+2.07 


Language Arts Mechanics 


21 


0.54 


0.26 


21 


1.77 


1.32 


+1.23 


+1.80 


Language Arts Useage 


21 


0.48 


0.22 


21 


1.56 


1.91 


1.08 


+2.60 


Spelling 


21 


0.65 


0.29 


21 


0.64 
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-0.01 


0.00 
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SA = strongly agree 

A = Agree 

DK - Don't know 

D = Disagree 

SD ^ Strongly disagree 
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STUDENT QUESTIONNAIRE 



NAME 

DATE _„ 

DIRECTIONS: 

The nurpose of this questionnaire is to help the school evaluate the effects 
of the Individualized Learning Center upon some of your beliefs. There are no right 
or wronn answers to the questions. Please read each question carefully before 
either circling T (true) or F (false). Your answers will be kept private. 

1. Planning only makes a person unhappy since plans hardly ever work out 

anyway. T F 

2. When one is born, the success one is going to have is already in the 

cards, so one might as well accept it and not fight against it. T F 

3. Nowadays, with world conditions the way they are, the wise person lives 

for today and lets tomorrow take care of itself. T F 

4. Even when teenagers get married, their main layalty should still belong 

to their parents. T F 

5. When the time comes for a young person to take a job, that person 
should stay near his or her parents, even if it means giving up a good 

job opportunity. T F 

6. Nothing in life is worth the sacrifice of moving away from one's parents. T F 

7. The best kind of job is where one is oart of an organization, all work- 
ing together, even if one doesn't get individual credit. T F 

R. It is silly for a person to put money into a car when money could be 

used to qet started in business or for an education. T F 



Thank you for helping the school to help 
you by knowing some of your beliefs. 
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